Efficient Cancer Classification by Coupling Semi Supervised and Multiple Instance Learning

The annotation of large datasets is often the bottleneck in the successful application of artificial intelligence in computational pathology. For this reason recently Multiple Instance Learning (MIL) and Semi Supervised Learning (SSL) approaches are gaining popularity because they require fewer annotations. In this work we couple SSL and MIL to train a deep learning classifier that combines the advantages of both methods and overcomes their limitations. Our method is able to learn from the global WSI diagnosis and a combination of labeled and unlabeled patches. Furthermore, we propose and evaluate an efficient labeling paradigm that guarantees a strong classification performance when combined with our learning framework. We compare our method to SSL and MIL baselines, the state-of-the-art and completely supervised training. With only a small percentage of patch labels our proposed model achieves a competitive performance on SICAPv2 (Cohen's kappa of 0.801 with 450 patch labels), PANDA (Cohen's kappa of 0.794 with 22,023 patch labels) and Camelyon16 (ROC AUC of 0.913 with 433 patch labels). Our code is publicly available at https://github.com/arneschmidt/ssl_and_mil_cancer_classification.European Union's Horizon 2020 Research and Innovation Program through the Marie Skodowska Curie (Cloud Artificial Intelligence For pathologY (CLARIFY) Project) 860627Spanish Government PID2019-105142RB-C2

Proportion constrained weakly supervised histopathology image classification

Multiple instance learning (MIL) deals with data grouped into bags of instances, of which only the global information is known. In recent years, this weakly supervised learning paradigm has become very popular in histological image analysis because it alleviates the burden of labeling all cancerous regions of large Whole Slide Images (WSIs) in detail. However, these methods require large datasets to perform properly, and many approaches only focus on simple binary classification. This often does not match the real-world problems where multi-label settings are frequent and possible constraints must be taken into account. In this work, we propose a novel multi-label MIL formulation based on inequality constraints that is able to incorporate prior knowledge about instance proportions. Our method has a theoretical foundation in optimization with logbarrier extensions, applied to bag-level class proportions. This encourages the model to respect the proportion ordering during training. Extensive experiments on a new public dataset of prostate cancer WSIs analysis, SICAP-MIL, demonstrate that using the prior proportion information we can achieve instance-level results similar to supervised methods on datasets of similar size. In comparison with prior MIL settings, our method allows for ∼ 13% improvements in instance-level accuracy, and ∼ 3% in the multi-label mean area under the ROC curve at the bag-level.Spanish Government PID2019-105142RB-C2European Commission 860627Generalitat Valenciana/European Union through the European Regional Development Fund (ERDF) of the Valencian Community IDIFEDER/2020/030Universitat Politecnica de Valenci

Deep Gaussian processes for biogeophysical parameter retrieval and model inversion

Parameter retrieval and model inversion are key problems in remote sensing and Earth observation. Currently, different approximations exist: a direct, yet costly, inversion of radiative transfer models (RTMs); the statistical inversion with in situ data that often results in problems with extrapolation outside the study area; and the most widely adopted hybrid modeling by which statistical models, mostly nonlinear and non-parametric machine learning algorithms, are applied to invert RTM simulations. We will focus on the latter. Among the different existing algorithms, in the last decade kernel based methods, and Gaussian Processes (GPs) in particular, have provided useful and informative solutions to such RTM inversion problems. This is in large part due to the confidence intervals they provide, and their predictive accuracy. However, RTMs are very complex, highly nonlinear, and typically hierarchical models, so that very often a single (shallow) GP model cannot capture complex feature relations for inversion. This motivates the use of deeper hierarchical architectures, while still preserving the desirable properties of GPs. This paper introduces the use of deep Gaussian Processes (DGPs) for bio-geo-physical model inversion. Unlike shallow GP models, DGPs account for complicated (modular, hierarchical) processes, provide an efficient solution that scales well to big datasets, and improve prediction accuracy over their single layer counterpart. In the experimental section, we provide empirical evidence of performance for the estimation of surface temperature and dew point temperature from infrared sounding data, as well as for the prediction of chlorophyll content, inorganic suspended matter, and coloured dissolved matter from multispectral data acquired by the Sentinel-3 OLCI sensor. The presented methodology allows for more expressive forms of GPs in big remote sensing model inversion problems.European Research Council (ERC) 647423Spanish Ministry of Economy and Competitiveness TIN2015-64210-R DPI2016-77869-C2-2-RSpanish Excellence Network TEC2016-81900-REDTLa Caixa Banking Foundation (Barcelona, Spain) 100010434 LCF-BQ-ES17-1160001

Combining Attention-based Multiple Instance Learning and Gaussian Processes for CT Hemorrhage Detection

This work has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska Curie grant agreement No 860627 (CLARIFY Project) and also from the Spanish Ministry of Science and Innovation under project PID2019-105142RB-C22.Intracranial hemorrhage (ICH) is a life-threatening emer- gency with high rates of mortality and morbidity. Rapid and accurate de- tection of ICH is crucial for patients to get a timely treatment. In order to achieve the automatic diagnosis of ICH, most deep learning models rely on huge amounts of slice labels for training. Unfortunately, the manual annotation of CT slices by radiologists is time-consuming and costly. To diagnose ICH, in this work, we propose to use an attention-based multiple instance learning (Att-MIL) approach implemented through the combi- nation of an attention-based convolutional neural network (Att-CNN) and a variational Gaussian process for multiple instance learning (VGP- MIL). Only labels at scan-level are necessary for training. Our method (a) trains the model using scan labels and assigns each slice with an at- tention weight, which can be used to provide slice-level predictions, and (b) uses the VGPMIL model based on low-dimensional features extracted by the Att-CNN to obtain improved predictions both at slice and scan levels. To analyze the performance of the proposed approach, our model has been trained on 1150 scans from an RSNA dataset and evaluated on 490 scans from an external CQ500 dataset. Our method outperforms other methods using the same scan-level training and is able to achieve comparable or even better results than other methods relying on slice- level annotations.European Commission 860627Spanish Government PID2019-105142RB-C2

Blind color deconvolution, normalization, and classification of histological images using general super Gaussian priors and Bayesian inference

This work was sponsored in part by the Agencia Es-tatal de Investigacion under project PID2019-105142RB-C22/AEI/10.13039/50110 0 011033, Junta de Andalucia under project PY20_00286,and the work by Fernando Perez-Bueno was spon-sored by Ministerio de Economia, Industria y Competitividad un-der FPI contract BES-2017-081584. Funding for open access charge: Universidad de Granada/CBUA.Background and Objective: Color variations in digital histopathology severely impact the performance of computer-aided diagnosis systems. They are due to differences in the staining process and acquisition system, among other reasons. Blind color deconvolution techniques separate multi-stained images into single stained bands which, once normalized, can be used to eliminate these negative color variations and improve the performance of machine learning tasks. Methods: In this work, we decompose the observed RGB image in its hematoxylin and eosin components. We apply Bayesian modeling and inference based on the use of Super Gaussian sparse priors for each stain together with prior closeness to a given reference color-vector matrix. The hematoxylin and eosin components are then used for image normalization and classification of histological images. The proposed framework is tested on stain separation, image normalization, and cancer classification problems. The results are measured using the peak signal to noise ratio, normalized median intensity and the area under ROC curve on five different databases. Results: The obtained results show the superiority of our approach to current state-of-the-art blind color deconvolution techniques. In particular, the fidelity to the tissue improves 1,27 dB in mean PSNR. The normalized median intensity shows a good normalization quality of the proposed approach on the tested datasets. Finally, in cancer classification experiments the area under the ROC curve improves from 0.9491 to 0.9656 and from 0.9279 to 0.9541 on Camelyon-16 and Camelyon-17, respectively, when the original and processed images are used. Furthermore, these figures of merits are better than those obtained by the methods compared with. Conclusions: The proposed framework for blind color deconvolution, normalization and classification of images guarantees fidelity to the tissue structure and can be used both for normalization and classification. In addition, color deconvolution enables the use of the optical density space for classification, which improves the classification performance.Agencia Es-tatal de Investigacion PID2019-105142RB-C22/AEI/10.13039/50110 0 011033Junta de Andalucia PY20_00286Ministerio de Economia, Industria y Competitividad under FPI BES-2017-081584Universidad de Granada/CBU

Deep Gaussian processes for multiple instance learning: Application to CT intracranial hemorrhage detection

Background and objective: Intracranial hemorrhage (ICH) is a life-threatening emergency that can lead to brain damage or death, with high rates of mortality and morbidity. The fast and accurate detection of ICH is important for the patient to get an early and efficient treatment. To improve this diagnostic process, the application of Deep Learning (DL) models on head CT scans is an active area of research. Although promising results have been obtained, many of the proposed models require slice-level annotations by radiologists, which are costly and time-consuming. Methods: We formulate the ICH detection as a problem of Multiple Instance Learning (MIL) that allows training with only scan-level annotations. We develop a new probabilistic method based on Deep Gaussian Processes (DGP) that is able to train with this MIL setting and accurately predict ICH at both slice- and scan-level. The proposed DGPMIL model is able to capture complex feature relations by using multiple Gaussian Process (GP) layers, as we show experimentally. Results: To highlight the advantages of DGPMIL in a general MIL setting, we first conduct several controlled experiments on the MNIST dataset. We show that multiple GP layers outperform one-layer GP models, especially for complex feature distributions. For ICH detection experiments, we use two public brain CT datasets (RSNA and CQ500). We first train a Convolutional Neural Network (CNN) with an attention mechanism to extract the image features, which are fed into our DGPMIL model to perform the final predictions. The results show that DGPMIL model outperforms VGPMIL as well as the attention-based CNN for MIL and other state-of-the-art methods for this problem. The best performing DGPMIL model reaches an AUC-ROC of 0.957 (resp. 0.909) and an AUC-PR of 0.961 (resp. 0.889) on the RSNA (resp. CQ500) dataset. Conclusion: The competitive performance at slice- and scan-level shows that DGPMIL model provides an accurate diagnosis on slices without the need for slice-level annotations by radiologists during training. As MIL is a common problem setting, our model can be applied to a broader range of other tasks, especially in medical image classification, where it can help the diagnostic process.Project P20_00286 funded by FEDER/Junta de Andalucía-Consejería de Transformación Económica, Industria, Conocimiento y Universidadesthe European Union’s Horizon 2020 research and innovation programme under the Marie Skodowska Curie grant agreement No 860627 (CLARIFY Project).Funding for open access charge: Universidad de Granada / CBUA

Learning Moore-Penrose based residuals for robust non-blind image deconvolution

This work was supported by grants P20_00286 and B-TIC-324-UGR20 funded by Consejería de Universidad, Investigación e Innovación ( Junta de Andalucía ) and by “ ERDF A way of making Europe”. Funding for open access charge: Universidad de Granada / CBUA.This paper proposes a deep learning-based method for image restoration given an inaccurate knowledge of the degradation. We first show how the impulse response of a Wiener filter can approximate the Moore-Penrose pseudo-inverse of the blur convolution operator. The deconvolution problem is then cast as the learning of a residual in the null space of the blur kernel, which, when added to the Wiener restoration, will satisfy the image formation model. This approach is expected to make the network capable of dealing with different blurs since only residuals associated with the Wiener filter have to be learned. Artifacts caused by inaccuracies in the blur estimation and other image formation model inconsistencies are removed by a Dynamic Filter Network. The extensive experiments carried out on several synthetic and real image datasets assert the proposed method's performance and robustness and demonstrate the advantage of the proposed method over existing ones.Junta de Andalucía P20_00286, B-TIC-324-UGR20ERDF A way of making EuropeUniversidad de Granada / CBU

Introducing instance label correlation in multiple instance learning. Application to cancer detection on histopathological images

In the last years, the weakly supervised paradigm of multiple instance learning (MIL) has become very popular in many different areas. A paradigmatic example is computational pathology, where the lack of patch-level labels for whole-slide images prevents the application of supervised models. Probabilistic MIL methods based on Gaussian Processes (GPs) have obtained promising results due to their excellent uncertainty estimation capabilities. However, these are general-purpose MIL methods that do not take into account one important fact: in (histopathological) images, the labels of neighboring patches are expected to be correlated. In this work, we extend a state-of-the-art GP-based MIL method, which is called VGPMIL-PR, to exploit such correlation. To do so, we develop a novel coupling term inspired by the statistical physics Ising model. We use variational inference to estimate all the model parameters. Interestingly, the VGPMIL-PR formulation is recovered when the weight that regulates the strength of the Ising term vanishes. The performance of the proposed method is assessed in two real-world problems of prostate cancer detection. We show that our model achieves better results than other state-of-the-art probabilistic MIL methods. We also provide different visualizations and analysis to gain insights into the influence of the novel Ising term. These insights are expected to facilitate the application of the proposed model to other research areas.European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska Curie grant agreement No 860627 (CLARIFY Project)Spanish Ministry of Science and Innovation under project PID2019-105142RBC22University of Granada and FEDER/Junta de Andalucía under project B-TIC-324-UGR20 (Proyectos de I+D+i en el marco del Programa Operativo FEDER Andalucía)Margarita Salas postdoctoral fellowship (Spanish Ministry of Universities with Next Generation EU funds

Variational Bayesian Blind Color Deconvolution of Histopathological Images

2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.Most whole-slide histological images are stained with two or more chemical dyes. Slide stain separation or color deconvolution is a crucial step within the digital pathology workflow. In this paper, the blind color deconvolution problem is formulated within the Bayesian framework. Starting from a multi-stained histological image, our model takes into account both spatial relations among the concentration image pixels and similarity between a given reference color-vector matrix and the estimated one. Using Variational Bayes inference, three efficient new blind color deconvolution methods are proposed which provide automated procedures to estimate all the model parameters in the problem. A comparison with classical and current state-of-the-art color deconvolution algorithms using real images has been carried out demonstrating the superiority of the proposed approach.Visual Information Processing (Ref. TIC-116

Bayesian K-SVD for H and E blind color deconvolution. Applications to stain normalization, data augmentation and cancer classification

This work was supported by project PID2019-105142RB-C22 funded by MCIN / AEI / 10.13039 / 501100011033, Spain, and project P20_00286 funded by FEDER /Junta de Andalucía-Consejería de Transformación Económica, Industria, Conocimiento y Universidades, Spain. The work by Fernando Pérez-Bueno was sponsored by Ministerio de Economía, Industria y Competitividad , Spain, under FPI contract BES-2017-081584 . Funding for open access charge: Universidad de Granada / CBUA, Spain.Stain variation between images is a main issue in the analysis of histological images. These color variations, produced by different staining protocols and scanners in each laboratory, hamper the performance of computer-aided diagnosis (CAD) systems that are usually unable to generalize to unseen color distributions. Blind color deconvolution techniques separate multi-stained images into single stained bands that can then be used to reduce the generalization error of CAD systems through stain color normalization and/or stain color augmentation. In this work, we present a Bayesian modeling and inference blind color deconvolution framework based on the K-Singular Value Decomposition algorithm. Two possible inference procedures, variational and empirical Bayes are presented. Both provide the automatic estimation of the stain color matrix, stain concentrations and all model parameters. The proposed framework is tested on stain separation, image normalization, stain color augmentation, and classification problems.CBUAJunta de Andalucía-Consejería de Transformación Económica, Industria, Conocimiento y UniversidadesFamily Process Institute BES-2017-081584Universidad de GranadaEuropean Regional Development FundMinisterio de Economía, Industria y Competitividad, Gobierno de EspañaAgencia Estatal de Investigación P20_0028